[Music]
all right so we'll go ahead and get
started so today we're gonna be talking
about our TX GI this is the new lighting
technique that we're using for global
illumination unity so this is a
collaboration between Nvidia and unity
bringing real-time realistic lighting to
unities high definition render pipeline
or HDR P in an artist friendly way with
our TX GI
this is global illumination that runs
everywhere and benefits from hardware
acceleration on NVIDIA r-tx GPUs so
here's an outline of what we'll be
covering today first I'm going to go
over a ray tracing and light theory so
we're all on the same page with the
theory but that underlies our TX GI then
I'll explain a little bit about how our
txg eye works using using pictures and
then kuba will show you a live demo of
our TX GI in the unity editor and so
we'll end with some specific advice for
how to construct and light scenes in
unity for ray tracing and our date our
TX GI and show some results on demo
scenes so ray tracing so this is a
pretty simple scene setup there's a
light at the top which I'll call an
emitter it's a simple box geometry with
a yellow wall on the left a gray sphere
on the floor and then a camera view and
so the diagram is showing light from the
emitter being cast in all directions and
it hits the yellow wall on the left and
bounces yellow light towards the camera
and it hits the gray sphere on the
bottom and bounces gray light towards
the camera and this is how this is how
we see color in real life this is actual
light transport and this is what when
you render it we call direct light so
because the grey sphere is on the ground
at Casa shadow and so what that means is
that it blocks light from the emitter
that would have hit the ground and this
is what we call a direct shadow because
the sphere is blocking visibility
directly from the emitter to the ground
so shadows have historically been
computed with shadow maps but now that
we have ray tracing ray tracing can
produce perfectly sharp point light
shadows and correct soft area light
shadows in real time so global
illumination realistic lighting requires
not just direct light but also indirect
light this is
bounce light that has scattered off of
multiple objects possibly before hitting
the camera and this accounts for a
phenomena like mirror reflection glossy
reflection diffuse into reflection which
you'll see a lot of in today's talk and
you know caustics refraction like you
name it most of the all of all of these
lighting phenomena are due to multiple
bounces of light coming from the emitter
and hitting the camera I talked about
direct shadows in the previous slide
there are also indirect shadows this is
when an object blocks indirect light
that would have hit the camera this is
also known as ambient occlusion so you
can see an example of this on the left
the red X here on the left is blocking
what would have been is creating an
indirect shadow from the ball from that
reflected light from the blue wall so
the highest quality way to compute
global illumination is with ray tracing
this is how you render movies any
everything with offline rendering so you
probably know about light Maps which is
how most games handled indirect lighting
until now light maps and light probes
are ray traced pre-baked so they're so
it's before you actually run the game in
real time but now now that ray tracing
can run in real time we can replace
these baked light maps and light probes
with dynamic lighting so I'll talk a
little bit about how that works in you
today so ray tracing immunity provides a
key advantage to creators and this is
because it allows developers to make and
dynamically alter content in real time
that looks photorealistic for maximum
photo realism uses ray tracing to
provide reflections like I mentioned
earlier and accurate dynamic
computations for global lighting so some
key elements here ray tracing is added
to unity with a plug-and-play
architecture so what that means is that
you can mix and match different ray
trace effects to get shadows reflections
refractions what-have-you and you can
mix and match them also with rasterized
effects so it means that you have full
flexibility in creating the effect that
you want in whatever scene or game that
you're trying to set up and in addition
a unity also exposes an additional layer
on top of an additional layer of ray
tracing on top of what's already
implemented in unity so that then you
can implement your own ray traced
effects on top of what's available with
having to worry about the underlying
hardware API or the underlying hardware
or the ray-tracing API that you're using
and all this is available in the unity
20 19.3 preview and on nvidia r-tx GPUs
based on a new Turing architecture all
these ray tracing effects to arts are
accelerated so you can use more of them
and you can use them at higher quality
so this is the the full lineup of Nvidia
r-tx GPUs and so the ray tracing
performance
scales across this entire lineup from
our TX 2060 all the way up to Titan r-tx
to demonstrate that let's just do a
little little back-of-the-envelope
computation here we'll pick one of the
sort of mid mid-range ones here 2070
super and so say you want to retrace
your content in 4k at 120 Hertz oh sorry
about that
so you want to retrace your content in
4k 120 Hertz so if you want ray-traced
reflections raytrace shadows and using
our TX GI
then you have eight million reflection
rays at one particular rays at four per
pixel using our TX GI you have four
thousand samples with 256 rays per each
sample and then for shadow Ray's for
each of those all at 120 fps so you
multiply all that together and you get
roughly that comes out to six agree rays
per second the our TX 2070 super is
capable of seven Giga raise per second
at peak ray-tracing throughput now it's
important to know here this is this is
theoretical throughput of ray-tracing
obviously the actual performance you're
going to get is going to be dependent on
your scene because you have to shade all
of those rays and the shading cost is
going to be dependent on the material
model you're evaluating so in our actual
research code we tend not to push things
quite so high but it shows you that
there's the horsepower is available on
the GPU for doing ray tracing in real
time so now we'll talk about our TX GI
and what it's useful for and how it
works so why why do you want our TX GI
well Archie xgi is going to completely
change workflow around lighting if you
use our TX GI and unity
you get immediate lighting feedback on
all platforms where the high-definition
rendered pipeline runs so what this
means is if you move an object or you
move a light the lighting changes and
the GI updates instantly not in five
minutes not in 20 minutes but instantly
you don't have to redo parametrizations
you don't have to try to fix up corners
or sort of retune things there's almost
no light or shadow leaking using our TX
GI even with no tuning and we'll go and
we'll go into later in the talk we'll go
into how to tune it for maximum image
quality on top of that so here's some of
the lighting effects that you'll get
with our TX GI I described earlier how
bounce lighting in a scene in direct
light creates all of these effects so
there's more there are more phenomena
computed by RG x GI that are up here on
this slide but here are four of the
major effects so let's take let's take a
miss of shadows for example so there's
my shadows what that means is that if
you have say a large glowing object like
a bonfire or something that will with
our TX GI that will act as a true light
source so it will cast it will cast it
will cast indirect light and you'll get
indirect shadows off of that emissive
object for free it will not be it
doesn't have to be evaluated as part of
your light loop its emissive so it just
gets evaluated normally in the material
and our texture will give you indirect
lighting so having gone over a bit of
what you get with our TX GI I'll
describe briefly how it works
so our TX GI is a probe based technique
and I visualized the probes on this
slide normally of course they wouldn't
be visible and so you'll notice that all
of these probes are sort of this sort of
light orangish color and that's because
most of the indirect light in the scene
is orange color because of those walls
so the probes are updated every frame
using ray tracing and so rays are traced
from the probes along this live this ray
which I've colored orange to show the
color of the light and when they hit a
point in the scene that point is lit
with direct light which I'm showing with
this white arrow here and then that
value is stored in the probe to provide
indirect bounce lighting in the scene so
that's one bounce of global illumination
but actually you get Mol
ball bounces of global illumination
because when this ray shot from the
probe hits the scene that point is
shaded with the existing probes and so
once that happens once you get two
bounces then every frame three bounces
and that eventually that converges
towards what would be the correct
infinite bounce lighting solution that
you would see in a rate race in a fully
path tracing so one thing you'll look at
here if you if you'll notice if you look
here is so these probes are completely
dark
that doesn't seem correct but actually
if you look at where they are these
probes are sticking out of this wall
they are inside the geometry they don't
see any light so it is correct for them
to be dark but the other thing that's
interesting is that there's no leaking
of that darkness onto this wall there's
no sort of shadow leakage from these
probes that's because our txg I
correctly removes these probes from the
shading computation when you render the
scene from the point of view of the
camera but you know it would still be
nice to use those probes for lighting
because you know the more the more
samples the better that's this YRT xgi
ships with a probe optimizer and what
this does is it dynamically adjusts the
probes over any geometry that you
flagged as static so that you get more
use out of the probes for lighting the
scene in real time and so you can see
all the probes that used to be in the
wall are now adjusted outside and are
being usefully used for lighting
competition but actually because we have
ray tracing we can do even more so
there's the sort of big dark spot in the
frame here that's actually a mirror and
so we have ray tracing so we can give
you real mirror reflections and one
thing you'll notice if you look in this
mirror reflection is that we have these
sort of spheres up here that are that
are being reflected in the mirror those
are also glassy but they're dark so
we're not getting any what are called
second-order reflections this would be
the light reflected from this Fiat from
this sphere to the mirror and then back
to the camera and so you could compute
the second-order reflections with ray
tracing but it's even more performant to
use our TX GI to approximate the
second-order reflect
or if you wanted to raytrace them you
could approximate the third or fourth
order reflections and this and this
gives you the sort of a full like multi
bounce not only diffuse reflection but a
glossy reflection in your scene and this
is approaching the complete lighting
solution you'll get using our TX GI with
unity all right so that's so much for
intro I'll hand it over to Cuba who's
going to give you a live demo on the
editor take it away
Thanks give me just a second to switch
to display so I'm gonna show you two
projects the first one is the Shanghai
streets project from one of the icons
back that was collaborating on this is a
scene that's an outdoor scene that is
made to be used on logger power devices
and basically we'll we'll try to light
it here this is the extent of the scene
you can see right now it's only lit with
the hgri
you can see that once I kill it it goes
black once I bring it back it was
brighter but you could ask well how is
this actually lit right up so if we
disabled articles UI this is how it
would look if it was only lit by default
I mean probe and reflection probe so the
default
I mean probe the only thing it sees is
the sky so you can see that in all the
other ways there's plenty of light
because that surface for example
directly sees the bright area in there
in the Sky box and the Nibelung RTX GI
gives us this nice occlusion in in
alleyways let's let's see where are the
probes actually placed so right now
we're running at a pretty low resolution
of just ten by five by ten probes and
you can see here we we have a volume of
of that many probes and volume extents
are wrapping the the entire scene but we
easily have headway on this machine to
to bump it up to get a higher quality
and we can bump it up even further here
when it comes to the amount of rays that
we will shoot per probe in each frame
now let's take a look at some
directional lighting so we added this
sunlight coming from there let's take a
look at the street level
that's what does it give us so here you
can see that the direct light is hitting
that brick wall and here on the street
you are getting pretty nice red bounce
from that light and if we disable and
enable again let's take a look at
lighting only view so here we're
removing all the base textures and only
showing the lighting output so that you
can appreciate that part of it a bit
more so let's take the light
and see how the bounce changes now the
question is how time-consuming is it to
set it up well actually in this case
it's not at all the only thing we did it
was to bring the de prefab in that
contains already the script that
schedules all the tracing and it
contains a volume component where you
can control the resolution and also it
has controls for this size so you can
see here that if I change the size to
something small right now the the volume
only covers a very small part of the
scene and the lighting is broken and the
rest if I undo and that's you know size
the volume correctly then the lighting
it's it's fixed now do we need to
manually adjust any probes no as sundar
mentioned before all the visibility
information that is stored in probes
that you can actually check out here all
that information that start in those
probes allows us to not bother too much
about where the probes are placed which
also means that not only we can set it
up really quickly but we can do any
modifications to geometry like for
example moving this building here and
not move any probes all of this updates
now what is what is lagging behind a bit
is multiple bounces as animation before
since we're using the results from from
the previous computation went racing
against geometry of course it takes a
tiny bit of time to propagate the
lighting but even that
for slow enough changes is not really
noticeable now let's switch to the other
project that we have ready
this is an example of an indoor scene
it's from the VFX demo from GDC and in
here we have a bunch of lights set up
let's take a look actually at what do we
have in the probes themselves just make
them a bit smaller you can see that
there's actually quite a decent amount
of bounce light in here I'm gonna break
it down for you a bit so that you can
see which part of the the final frame is
coming from where so if we just take a
look at lighting only this is all the
lighting that that we were getting on on
opaque surfaces then if we will turn off
in direct indirect actually lights quite
a big part of even that small corner now
the rest of the lighting comes from
those monitors they're set up actually
in in a dual matter if you will there's
both the PC screens that are just an
emissive material
so whenever for our probes whenever
we're shooting race if we take a look at
the probes again those probes that are
closer to those screens have a pretty
high chance of hitting those those
screens and propagating that that
lighting further so that's what happens
here let's just try to adjust the
lighting now you'll see that when I
adjust this not all the lighting changes
that's because we also have some lights
set up to to light volumetrics in a
direct way I'll just turn that off for a
second just so that we can isolate it
so this is all the lighting that we were
getting from the emissive surfaces that
that are in that scene and that means no
additional shading cost and you were
getting that's under mission before
indirect shadows from from those probes
so one thing is actually using it fully
real time on your users devices but the
other way you could use it is actually
to have an instant feedback in the
editor and then store the result for
using in the player right now in this
prototype we're supporting just one
volume but concurrently we're working on
supporting multiple volumes so that for
different areas you can decide on
resolutions on update rates you can post
the different volume so that you can
easily scale it to two bigger scenes but
as you could see in that outer scene
even for a reasonably sized small city
block it it works just fine and there's
a lot of headroom even on this laptop
for a casting many more rice and a lot
of headroom for optimization for us as
well now I'll hand it back to Center
so we did both of those just a second
okay come on properly
sorry rubric alright
thanks cool thing alright so for the
rest of the talk I'm gonna cover a
little bit of ray-tracing advice for
artists this is advice for making your
scenes be beautiful and performant using
our TX GI so the first thing I want to
talk about is volume placement so RT x
GI computes global illumination multiple
bounces of light this means that you get
a benefit from having probes everywhere
even in places where the player can't
see because then lighting can propagate
through those places to places where the
player can see for more realistic
lighting so what that means is you want
to basically have your volume big enough
to encompass your urine or your entire
scene and so there are some other there
are some other guidelines here I won't
go through all of them but basically the
the takeaway from this slide is when
you're initializing that probe volume
just like kuba was doing you want to
make sure that not only is it big enough
to encompass your scene but you have
enough probes in there to get a good
sampling rate across spatially across
your scene so probes roughly you know
every every one to two meters for human
scale scenes and you can get away with
even larger spacing on sort of larger
level size scenes so talk a little bit
about tuning the volume so the most
important control for tuning the volume
is what we call the self shadow bias
this is the bias that controls whether
or not a probe shades the point that it
thinks it can see or not and if you and
you increase that if you increase this
self shadow bias you can get rid of any
small lingering light leaks and sharp
corners that might be there might be
there and if you can get away with it
lower then it creates more contrast in
the lighting so here's what that looks
like
so this is with the self shadow bias off
and what you can see is you get this
really sort of this really ugly result
from the GI because there's sort of a
it's the the visibility is being sampled
directly at the point where the mean
distance result is sort of has it's has
it has its mean so you get this weird
sort of pattern you can see some of it
over here too
this is with self shadow bias that's too
high so you can see that all of those
sort of weird liking results are good
looking results are gone but the
lighting everywhere else looks really
flat
we've lost all of that sort of nice sort
of ambient occlusion detail that was on
the sort of edge of the roof here and
then also back here on this wall so this
is with correct this is with correct
self shadow bias so you can see we've
gotten back this ambient occlusion
detail in sort of an on the roof and on
this wall and the result back here is
correctly dark and it should be also so
you can also rely on the optimizer as I
showed earlier to improve coverage and
get sort of more more lighting samples
in your scene you'll also be able to
optionally move the probes if you need
to but this is sort of maybe maybe
around your hero object but this is sort
of rare you shouldn't need to use this
for your normal workflow so I'll talk a
little about geometry so again you know
real light real light bouncing realistic
realistic global illumination you want
to make walls with real thickness and
I'll show you an example of why so this
is a sort of simple scene that
illustrates this concept there's a
closed box that's what this is with a
closed completely closed room so dark on
the left and a room with one open door
on the right and there's a wall here
that divides the two of them we're gonna
look at it from the point of view of
this camera so all the walls here are
infinitely thin so if you look at this
you'll see that on the wall that divides
oh it's very dark on the wall that
divides the two scenes there are these
sort of shadowing artifacts due to the
fact that this wall is infinitely thin
and so I'll circle them a little so you
can still you can see them a little bit
better and so if you make the wall
thanks google
little bit be easy to see be easier to
see afterwards but so if you make the
walls properly have actual actual
thickness then the lighting computation
proceeds as it should and those
artifacts go away and in fact the
illumination generally even the rest of
the scene improves also so here's the
final view with correctly thick walls
and so you get no artifacts on the
dividing wall correct ambient occlusion
in the corner and then sort of powerful
blob bounce sliding on the ceiling from
this from the outside as you should and
so here are some other notes for for
optimizing your geometry for quality
using single sided geometry is best
because I've been talking about the the
probe optimizer that will adjust the
probes that optimizer relies on back
faces and SATA geometry to be able to
adjust probes through walls efficiently
and I'll talk a little about this a
little bit more later but larger
emissive missives like kuba was showing
you lighting from in the demo having
larger ones will look a lot better than
having smaller ones because they're
easier to sample with ray tracing all
right now performance this may be the
most important piece of advice in the
talk and the reason is because the other
the other advice that I'm giving you
will help improve image quality with our
TX GI but this slide will help improve
performance not only with our TX GI but
with any technique that uses ray tracing
at all so basically what all of these
bullet points derive from is the fact
that to accelerate ray tracing the API
will divide your geometry spatially
basically will create bounding boxes
around it so just to take an example of
six take an example say you have a
triangle that is really long and thin
the bounding box access oriented that
you would have to create around that
triangle has high volume relative to the
actual surface area of the triangle what
that means is that a lot of rays will
hit the bounding box and not a lot of
rays will hit the triangle and this is
bad for performance so visually what
does that look like this is bad geometry
these triangles are huge relative to all
the other triangles in the scene there
are triangle fans up here all of these
really long thin triangle share of
vertex there are more triangle fans here
this is exactly what you don't want this
leads to bad ray-tracing performance
this by contrast is good geometry so if
you look at the floor here all the
triangles are laid out very neatly in
this grid like pattern
they're all exactly the same size there
are no triangle fans or sort of enormous
triangles bucking the curve so this is
this is good geometry for ray tracing
performance alright so a little bit
about lighting in the past without
real-time global illumination it's easy
to fall back on over lighting a scene by
placing a lot of direct lights to make
up for the light that's lost in bounced
lighting you no longer have to do this
in a realistic scene about 25 to 50
percent of the light should be coming
from the indirect bounced lighting and
so this has some interesting
implications what it means is you know
there are some scenes that you could
light using just the sky or just the sky
with one or two lights pointing at
certain certain places where you need
more light and so you know you want to
let when you went into and when you and
when you're letting a scene with either
the sky or like one of these big
directional lights you want to let the
light pick up the color for you pick up
the color of the indirect bounce so
color the materials and keep the light
neutral or color the light and keep the
materials neutral so this is scene
that's from the demo that the Kouga
showed you and I want you to see right
here this really nice sort of red bounce
lighting that we're getting from from
the opposite wall and so this is white
light coming into the scene and giving
us red bounce lighting from materials
and so this scene looks really good
because it's dominated by this red hue
so a little bit more on materials so we
showed the little bit of this in the
demo using emissive x' to get zero-cost
lights and I'll show you a result of
that another result of that in a second
but you want a favor sort of dominant
hues when you're using this technique
because if
you have sort of too many different
saturated colors they can interfere in
ways that create great bounce letting
and cancel each other out one of my
favorites one of my favorite examples of
this is if you have green light green
bounce light that reflects off of a red
wall that result is black so you get no
light you get you get no indirect bounce
light coming from that at all so here's
our Shanghai seen that that kuba showed
earlier and so this is sort of a you
know exaggerated haunted-house effect
but it's important to note that all of
the lighting in this scene is coming
from those emissive walls there are no
light there are no other lights in the
scene and so you can see that you can
see sort of in the streets and around
the sort of you know nooks and crannies
of this scene here there's a lot of
light coming in through those emissive
walls and so this is sort of the this is
something that you can leverage to
create really realistic looking scenes
using art xgi okay so I want to conclude
with one I think really salient examples
showing the visual impact of this
technique this is a scene from our demo
and this has you know as kuba mentioned
this has live the lights coming from
lights coming from the monitors which
are which are emissive as well in
spotlight it has sort of volumetric fog
and unity but there's not any there's no
bounce lighting here this is that same
scene with the bounce lighting from
Artie xgi and so I'll flip back and
forth a couple times so you can really
see the difference and so one thing
that's important to note here is when we
give these before and after images we're
not comparing to pre-bake solutions like
light maps or light probes and the
reason for that is because this is a
real-time GI solution just like just
like kuba showed in the demo you can
move any of these and missive services
or this light around the lighting
updates in real time so we're comparing
to basically pre like previous previous
real-time previous real-time lighting
which would have been basically only
your direct light and your volumetric
sand and everything else that setup in
unity and so for reference here is where
here's where the probes are in the scene
so this is
similar to the volume that kuba showed
you earlier and one reason I really
liked this scene is because it shows
multiple different effects from unity
you have sort of partly like a particle
effect going on here you have the
volumetric fog you have obviously the
direct lighting and the emissive
surfaces combined with bounced lighting
indirect effects that you get from RT
xgi and they look really really good
together
all right thank you very much we can
take questions
[Applause]
we have two microphones one on each side
so please don't be shy and ask us
anything oh yeah thanks for the talk
I'm particularly curious about using the
editor enhancements of real time editor
workflows and then shipping to you know
static players without the ITX hardware
how does that compare to like mapping
and pre computer-ji in terms of
performance do you have any information
on that the biggest benefit that you'll
get from this is the amount of memory
that is used we can actually switch it
back to the demo - I'll do that when in
a second but something itself is it's
fairly cheap it is more expensive than
something just by probes because we also
need to evaluate visibility however it
works well on on fairly low power
devices we still need to profile you
know across all the ranges so stay tuned
on that but definitely the the workflow
where you break out the data that you
see is it's really powerful and what is
also actually quite quite neat let me
just switch back and show you something
directly also that the update rate can
scale across hardware so if you do your
development with instant updates then
you can sort of set you can set the
update rate lower for like lower
performance hardware or even fall back
on a baked GI solution which will sort
of still build still still be you know
better than better better than previous
probes because it has the visibility
information so you still won't get those
leaks for example so we're back in the
scene and you can take a look here that
in this scene right now we're using
twelve mega bytes for the ray tracing
acceleration structure that you will not
need we're using that much memory for
the trace output and then the only thing
that
will actually need on your device is
this texture 250 kilobytes and this
texture 500 now this is what the
settings that I haven't bumped yet but
it was just bump it to what we had
before and now you can see that together
it's three megabytes to to light that
scene so when it comes if you would
compare that with lighting it with light
maps where you know it's it's it's not a
big scene but it has a really big
surface area if you take into
consideration all the all the elements
in it that would be a few 2k textures so
calculating that in the editor would
take take a while even what did you
progressive line mapper and then
actually shipping it to the devices
would take much much more memory the
other thing that on the optimization
kind of angle if you would see what's
actually in our acceleration structure
it's actually only those bigger pieces
of geometry here that that affect the
bounce in a big way everything else so
all the all the detail that's sprinkled
around its just picking up indirect
lighting without needing to be in the
acceleration structure so also depending
on the hardware that you'll be running
the editor in you'll be able to dial it
like this for the for edit time so
that's you know in the beginning you can
get the mood perfectly with really good
performance and if you really need
everything in the acceleration structure
to get bounce even from the small pieces
then you can add it and just let it bake
out one more follow-up question is this
only HD on the pipeline or is their plan
support for everything currently it is
but there's at least four using the
result that the static there is nothing
preventing it from being used in the
universal rather pipeline so we'll
definitely look into that yeah we can
see how that would be useful so for sure
alright thank you
you
[Music]